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DATA COLLECTION TECHNIQUES IN VOCATIONAL EDUCATION RESEARCH: 



An Introduction 

Gary E. Moore 
Louisiana State University 



Research 1n vocational education has been criticized as being Isolated 
from other academic disciplines (Matthews and Campbell, 1983). The 
research techniques and methodologies used 1n disciplines such as 
psychology, law, and sociology have largely been Ignored 1n vocational 
education according to Matthews and Campbell* They urged the profession to 
look at what other academic disciplines are doing 1n the research arena. 

One area of vocational education research that appears to be stagnate 
and out of touch with other academic disciplines 1s that of data 
collection. The mall questionnaire appears to have dominated all other 
forms of data collection 1n vocational education research. In a study of 
vocational education doctoral dissertations bin Yahya and Moore (1984) 
found that over 70 percent of the dissertations were survey research and 
depended primarily on the mailed questionnaire as the data collection 
device. Well over half of the research studies conducted 1n agricultural 
education during the past decade used mailed questionnaires (Mannebach, 
McKeena & Pfau, 1984). 

The merits of using the mall questionnaire have been discussed by 
Orllch, et. al. (1975), Som (1973), and Raj (1972). However, Frankel and 
Dutka (1983), Maclean and Green (1979) and Meyer (1971) have pointed out 
many serious drawbacks 1n using mailed questionnaires. Other disciplines 
utilize a wide array of data collection techniques such as personal 
Interviews and the telephone. In view of the criticism of Campbell and 
Matthews 1t appears the profession could benefit from re-examining the 
merits and drawbacks of using mailed questionnaires along with an 
examination of alternative data collection techniques which could be used 
1n vocational education research. 

Purpose of the Symposium 

The primary purpose of this symposium 1s to acquaint (or re-acqua1nt) 
vocational education researchers with four techniques for collecting data 1n 
vocational education research. The merits and shortcomings of each of the 
four techniques will be critically examined. The four techniques are: 

1. The Mall Questionnaire 

2. The Telephone 

3. The Personal Interview 

4. The Audio Cassette Tape 
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For each of the techniques the presenter will briefly review what the 

literature has to say about the technique and then the presenter will share 

his/her personal experiences 1n using the particular technique. Each of the 

techniques has been used by the presenters 1n conducting research 1n 
vocational education. 

If a profession 1s to grow and Improve 1t should be receptive to new 
Ideas. While three of the data collection techniques presented 1n the 
symposium may not be new, they are not commonly used in vocational educaticn 
research. As a result of attending this symposium, researchers may be more 
likely to use the data collection techniques discussed. The quality of the 
data collected 1n vocational education research and the resulting research 
findings may be Improved. 

The Intended outcome of this symposium 1s to Improve the quality of research 
1n vocational education. 
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A Review of the Techniques for 
Conducting Mail Survey Research 
Introduction 

The survey tends to be the standard method of evaluation 
within education circles (Arbuckle, cited in Horowitz & 
Sedlacek, 1974) with the mail questionnaire being the 
prevalent survey (Parten, cited in Horowitz & Sedlacek, 
1974). Hlllestad (1977) identifies the survey as a valuable 
exploratory tool for finding out what people know, what 
people like and dislike, what they think, what has taken 
place, and what is happening now* However, both Humphries 
(1983) and Hillestad (1977) contend that even though mail 
questionnaires are among the most frequently used tools of 
educational research, they are also among the most abused. 
Humphries (1983) offers two reasons for the misuse of the 
questionnaire: "(a) researchers fail to see the place of this 
data-gathering technique in the overall picture of research 
procedures and (b) researchers underestimate difficulties of 
constructing and using high-quality questionnaires" (p. 5). 

Hoskins (1976) states that most questionnaires seek 
information that can be easily obtained elsewhere, request 
trivial data, or ask questions that are too broad* However, 
this situation can be eliminated if the researcher will pay 
a t ten t ion to the following basic principals: (a) the f unct i on 
of the questionnaire is to gather information for specific 
purposes and (b) the researcher should not try to use a 
questionnaire if another research tool is more functional and 
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practical (Hoskins, 1976) . Sudman & Bradburn (1984) state 
that "when mail procedures are used appropriately, they can 
be expected to produce results ranging from almost as good as 
to substantially better than those that can be obtained by 
more costly methods" (p. 33). Let's look at the advantages 
and d i sad vantages that are associated with the mail survey 
techn i qua . 

Advantages and Disadvantages of M§ii Survey B£§£§££b 
Advantages of M<Ui Survey 

Less bias. Interviewer bias is eliminated on questions 
that could be sensitive or embarrassing when presented by an 
interviewer (Pride, cited in Cote & Grinnell, 1984). 

L^ss CostiXi The mail questionnaire technique is less 
costly than other techniques such as the personal interview 
that calls for recruiting, training, and supervising 
(Hochstim & Athanasopou los , 1970; Sudman and Bradburn, 1984). 

Respondent reflection and convenience. Respondents can 
decide how much time to spend filling out the questionnaire 
(Sudman & Bradburn, 1984) and they can complete the 
questionnaire at their own convenience ( E ige lberner , cited in 
Horowitz & Sedlacek, 1984) 

T^rne conservation^ Mail surveys can be used to gather 
data quickly (Cote & Grinnell, 1984). They can also be 
distributed so that all respondents receive them at about the 
same time (Humphries, 1983). In addition, the amount of time 
and energy required on the part of the researcher is less 
(Plog, cited in Horowitz & Sedlacek, 1974). 
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§£§Qdardized gH§L§ tj.onnai.re ^ * respondents receive the 
same questionnaire presented in the same way (Hillestad, 
1977) . 

Ease of tabu l_a t^on . I f quest i onna i res are we 1 1 
constructed, tabu 1 at ion is eas i ly completed through 
electronic means. M A tightly-structured questionnaire can 
provide comparable data for rapid analysis" (Hoskins, 1976, 
p. 12). 

Wide Geograghiga 1_ Qis tribu t^on^ A mail survey enables 
one to reach large samples that may otherwise be unavailable 
(Hillestad, 1977; Hoskins, 1976). Distant populations and 
individuals who are too busy to be interviewed may also be 
reached (Cote & Grinnell, 1984). In addition, mobile 
subjects may be followed without prohibitive cost (Hochstim & 
Athanasopoulos , 1970) . 

Of course where there are advantages to any research 
technique, there are also disadvantages. The following 
discussion provides some disadvantages of mail 
quest ionna ires . 
disadvantages of Mail. Survey 

Inaccurate Ma^l^ng lists. Mailing lists may be 
incomplete or inaccurate; or questionnaires may not be 
forwarded to new addresses (Lockhart, 1984). 

N2Qz£§L§E2Q§§L SQ^ inaccurate answers^ Non-response and 
the inability to check responses are two of the biggest 
problems in mail survey research (Kerlinger, cited in Cote & 
Gr innel 1 , 1984) . 
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Lack of Sgon tane j_t£ of answers. Respondents have more 
opportunity to consider their responses thus eliminating 
spontaneity of answers (Moser & Kalton, cited in Humphries, 
1983) . 

item dependence. No item can be considered independent 
of others because the respondents can read all questions 
before answering a questionnaire (Moser & Kalton, cited in 
Humphries , 1983) . 

Method of comgl_e tion^ Researchers cannot always be sure 
the questionnaire is being completed by the intended 
respondent (Humphries, 1983) or that the respondent is 
honestly answering the questions (Sudman & Bradburn, 1984). 

iiEiElieiJlY of guest^ons . Mail questionnaires do not 
allow the depth in questions that other methods allow such as 
the personal interview. The questions on a mail survey must 
be simple and straightforward (Hillestad, 1977). 

Susgj_c_[on toward gues tionnaires^ Some i nd i v i dual s may 
have little experience with questionnaires and may be 
suspicious of the study (Sudman & Bradburn, 1S84). 

Prejudice against guestionnaires . Berdie & Anderson 
(cited in Humphries, 1983) suggest that inaccurate 
information may be given by respondents distrustful of 
quest ionna ires . 

8H£§£i2QQ§i£§. Design 
A well designed questionnaire plays an important role in 
encouraging respondents to complete the questionnaire. 
Swisher (1980) is of the opinion that "the most effective 
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incentive to respond to a questionnaire is the quality of the 
questionnaire itself (p. 159). Hillestad (1977) identifies 
the design of the questionnaire as an important step to 
i ncrease response rates . 

Once all research techniques have been explored and the 
researcher decides that the mail questionnaire is the best 
technique to use for the problem, the first step in designinj 
the survey is to deve lop the quest ions . The f ol low ing 
guidelines are offered for developing questions and 
organizing the survey instrument: 

1. Visualize the respondents. Try to think of every 
possible way they may react to the question 
(Hillestad, 1977) . 

2. Arrange questions in either a psychological or 
logical order and group questions dealing with 
each aspect of the study (Hillestad, 1977). 
Swisher (1980) recommends sets of logically 
related content. The order of the questionnaire 
should maintain interest so the respondent will 

be motivated to continue (Demaline & Quinn, 1979). 

3 . Make apparent that the questi ons are re la ted to the 
purpose of the study (Hillestad, 1977) and that 

the need for the questionnaire is clearly evident 
(Demaline & Quinn, 1979). Keep in mind that 
the more the researcher knows how the research data 
are going to be used and the procedures that will oe 
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used in case of the unexpected, the more apprcpriate 
the questions will be (Swisher, 1980). 

4. Use an easy-to-answer format. (Hillestad, 1977). 
It should be physically and logically consistent 
(Swisher, 1980). Dillman (cited in Humphries, 1983) 
recommends that responses be made by circling rather 
than marking. This avoids confusion. 

5. Decide the form the questions should take, 
structured or unstructured response questions 
(open ended or closed responses) (Hillestad f 
1977)* Structured questions reduce the time 
required by the respondent and ease the 
researcher's analysis of the data. Structured 
questions also provide a series of response choices 
(Humphries, 1983) and pre-coded categories of 
answers. (Swisher, 1980). Check-off items 

are best where possible (Hillestad, 1977). 

6. Use simple language that is precise and clearly 
stated (Hillestad, 1977; Humphries, 1983). 

To the extent possible, all technical 
terminology should be removed (Demaline & Quinn, 
1979) 

7. Questions should apply to all respondents 
(Humphries, 1983). They should know the answers or 
have opinions (Demaline & Quinn, 1979). 

8. Use no "loaded" or "leading" questions 

(Hi 1 lest ad, 1977) , so-cal led " double -barrs led" 



9 

ERLC 



-6- 



12 



Ma i 1 Survey 



quest ions, (Humphries, 1983), or * tel 1 -me- 
every-thing-you-know-about" types of questions.* 4 ' 
(Hoskins, 1976, p 11). If the word "and" appears in 
the que st ion, ask yourse If if the quest i on is 
double-barreled (Babbie, 1973). 
9. Questions must be answerable (Hillestad, 1977), 
They should al 1 ow the respondent to answer 
quickly and easily (Swisher, 1980). Enough 
response options should be included (Humphries, 
1983) and for structured questions all 
answers should be available to the respondent 
(Hillestad, 1977) . 

10. Deal with only one topic in a question and have 
responses that relate directly to the question 
(Hillestad, 1977). 

11. Remove emotional overtones from the questions 
(Hillestad, 1977; Swisher, 1980) and 

avoid questions that are embarrassing to the 
respondents (Humphr ies , 198 3) . 

12. Threatening questions should not be used. 
Threatening questions are those to which society is 
perceived to have the "right answer* 1 (Sudman & 
Bradburn, 1984) . 

13. Avoid negative questions. Negative questions 
lend themse 1 ves to mis interpre tat ion (Babb ie , 
1973) . 
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14. Branching on mailed questionnaires should be 
kept simple* An example of branching is allowing 
the respondent to omit a question that 

does not apply* (Sudman & Bradburn, 1984), 

15. Prepare dummy tables of your anticipated responses. 
(Hillestad, 1977). Dummy tables will allow 

you to see if you will obtain usable data from the 
quest ions . 

Even though the literature could not single out "the 
attractiveness of the survey - as being a contributing factor 
to high returns, researchers such as Dillman, Dillman, and 
Makela (1984) in their Total Design M «thod have shown 
evidence that attention to detail makes a difference in 
return rates. Swisher (1980) states "the technical quality 
of the instrument is of paramount importance" (p. 164). 
Color was not supported in the literature as contributing to 
high returns, yet researchers tend to think that color paper 
and/or ink are more eye catching (Humphries, 1983; Swisher, 
1980). Many researchers recommend that the questionnaire be 
kept as short as possible (Hillestad, 1977) yet research 
shows that length of the questionnaire makes little or no 
difference in the response rate. ( Baumgar tner , 1984; Sletto 
cited in Hochstim & Athanasopoulos, 1970). 

Pilot testing is an extremely important step in the 
development of a survey instrument and is necessary to assure 
an effective instrument (Humphries, 1983). Isaac & Michael 
(1971) identify the failure to pretest a questionnaire as one 



ERIC 



-8- 

14 



Mail Survey 



of the common errors In questionnaire studies. This is the 
point at which many ambiguous and irrelevant questions can be 
detected. Dillman, (cited in Humphries, 1983) recommends two 
groups to pilot test the instrument, colleagues who are 
familiar with the study and potential users of the survey 
results. Pilot test respondents should be asked to identify 
troublesome questions, additional responses, and difficulties 
in completing the questionnaire. The pilot test can be used 
to help determine if the respondents can provide the needed 
information (Humphries, 1983). It may be helpful to analyse 
the pilot test data as the real data will be analysed. The 
pilot test process should be conducted as many times as 
necessary to obtain the best possible instrument (Hillestad, 
1977). Pilot testing can also provide information on the 
reliability and validity of an instrument. 

Questionnaire Procedures 
Once the questionnaire is designed, the next step is to 
design the procedures for mailing and receiving the 
quest ionnaire . The use of and the nature of the cover 
letter, the use of personalization and prenot i f icat ion, the 
timing, the type of postage, to use or not to use a aue date, 
the promise of anonymity, and the use of incentives are all 
dec is ions that a researcher has to make . Yet when it comes 
to singling out the effectiveness on increasing response 
rates for any one technique the literature shows substantial 
support only for persistent follow-up and monetary incentives 
(Baumgartner & Heberlein, 1984; Kanuk and Berenson, cited in 
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Powers & Alderman, 1982)* However, Dillman et al. (1984) 
show empirical evidence for a combination of procedures in 
their Total Design Method and other researchers find support 
for using various techniques when conducting mail survey 
research. Therefore, several procedures will be discussed. 
L*fLJiJi§LE5 and Persona ligation 

Several types of letters were identified in the 
1 i terature ; the cover letter, the endorsement letter, and the 
pre-not i f icat ion letter. A number of questions concerning 
such letters have been examined in the literature: should 
the cover letter be personally signed, does the status of the 
signature make a difference, should the cover letter be 
personally typed, and do pictures of the researcher help? 
Even though Humphries (1983 and Dillon et al . (1984) 
recommend personalization, it seems that personalization and 
status of the researcher alone do not tend to increase the 
response rate* Horowitz & Sedlacek (1972) find in their 
review of the literature that personalized signatures on the 
cover, letter do not guarantee higher response rates and in 
their own research find that neither status of the researcher 
nor type of signature resulted in a higher return rate. 

Rucker, Hughes, Thompson, Harrison, & Vanderl ip (1984) 
determined the effects of status by using two pictures with 
different dress styles and signatures that reflected status 
of position. They find that pictures did not induce early 
re turns aad may have actually depressed total re turns . 
Further they did not find support for status increasing 
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returns. Nor did Nitecki (1970) find support for sender's 
prestige increasing response rates in his study on the 
"Effects of Sponsorship and nonmonetary Incentive on Response 
Rate . - 

Rucker et al . (1984) identify five studies that support 
preno t i f icat i on and cite Linsky (19 7 5) who found 12 studies 
that supported preno t i f icat i on by letter, postcard, 
telephone, <r personal contact. All methods of 
preno t i f icat i on increase response rates w i th telephone be i ng 
the most effective. However, telephone preno t i f icat i on does 
tend to increase the cost of the survey cons iderabl y • I f an 
effective cover letter can be developed, the use of 
prenot i f icat i on may not be necessary. 

Humphries (1983) provides key points for developing a 
cover letter: 

1. The purpose of the study should be briefly 
stated. Graduate students should refrain 
from say i ng the re suits will be used in a 
dissertat ion or thes is . 

2. Let the respondents know they are important. 

3. Explain how the data will be used. 

4 . Sponsorsh ip of an organ i zat i on to wh ich most 
of the respondents belong should be gained 
and cited in the letter. 

5. Volunteer to answer questions and provide 

phone numbers to do so (Dillman, cited in Humphries, 
1983) . 
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6. Thank the respondent and offer the results. 

7. Further recommendations involve: 
a* Personal ized addresses 

b. A formal salutation 

c. A personally signed signature 

Along with the notion of prenot i f icat ion, the case for 
sponsorship tends to find support by researchers as a means 
for introducing the researcher to the respondent* 
Baumgartner & Heberlein (1984) find support in their review 
of the literature that government and university sponsorship 
tends to enhance responses while market research firm 
sponsorship may decrease response rate or at best does not 
affect it. 

XlSUDS Due Dates 

Timing and due dates are two other decisions that have 
to be made in the process of mailing anO receiving the mail 
survey. Altschuld & Lower (1984) attribute the high rate of 
return in their study on teacher evaluation to the topics of 
the questionnaire being of immediate cor.cern to the 
respondents . 

There seems to be conflicting data in the literature in 
regard to the effect of due dates. Swisher (1980) states 
that a due date is important because people tend to return 
material on or after a due date. Roberts et al. (cited in 
Baumgartner & Heberlein, 1984) find that for the initial 
mailing and first follow-up a three week deadline increase 
responses, but deadlines do not increase response rates for 
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further follow-ups* On the other hand, Humphries (1983) 
states that there is no empirical evidence to support the use 
of deadlines. Some researchers recommend that deadlines not 
be used because they increase non-response* 
AnoQYJSiiY 

The importance of anonymity tends to be stressed when 
conducting mail survey research (Demaline 6 Quinn, 1979; 
Hillestad, 1977). Baumgartner & Heberlein's (1984) report 
conflicting results in their review of literature, they 
conclude that it does not appear to be necessary to use 
anonymous procedures in order to obtain high results, yet the 
effects of anonymity are still not clear. However, they 
indicate that anonymity may be important in some instances 
such as with groups who do not want their identity known. 
Mali Procedures 

It is generally accepted that if you want something to 
be returned in the mail, you should include a ee 1 f-addressed 
stamped envelope. A se 1 f-addressed stamped envelope will 
impro/e response rates significantly (Hoskins, 1976). But, 
secondary questions are whether to use window envelopes and 
what class of postage should be used on both the return 
envelope and the sending envelope. Baumgarter & Heberlein 
(1984) state that the class of postage that obtains the best 
response rate has not yet been determined in the research 
literature. Humphries (1984) seems to think from his review 
of research that the type of postage on both the outgoing and 
return envelope affects response rates. He identifies 
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special delivery and hand-stamped return envelopes as more 
effective than postage-permit envelopes. In conflict, 
Rossmann & Astin (1974) report that the use of nonprofit 
outgoing postage, window envelopes, and business reply 
returns lower the costs and do not significantly lower the 
re turn rate • There tends to be no agreement as to the e f f ect 
of postage on the outgoing or return envelope . 

The use of incentives has been extensively explored as a 
technique of improving response rates. The only incentive 
that has consistently surfaced in the literature as improving 
response rates has been money (Baumgartner & Heberlein, 1984; 
Cote & Grinnell, 1984; Humphries, 1983). Baumgartner & 
Heberlein (1984) find that an incentive of a prepayment of 
$.25 to $1.00 is worthwhile, but only in the second mailing. 
Non-monetary incentives seem not to be as effective as money 
(Humphries, 1983). Nitecki (1978) finds that a bookmark as 
an incentive has no effect on response rate. However, Powers 
& Alderman (1982) find that although the promise of feedback 
has a significant positive effect on initial response rates, 
follow-up contact with non-respondents and a shorter 
quest i onna i re have more of an e f f ect . 
Cost Cutting i'gchnigues 

Even though various mail survey strategies may affect 
response rates, cost effectiveness is a factor that must be 
taken into consideration for most researchers. One of the 
primary reasons for using mail research in place of other 
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techniques, such as the interview, is the cost* Hochstim & 
Athanasopoulos (1970) show a contrast of mail cost per 
respondent of $6*55 each compared to $13*65 a for personal 
interview (p. 79) • Rossmann & Astin (1974) find that among 
most techniques, "cheapest is best" (p. 279) • They find 
that some cost cutting techniques, such as nonprofit outgoing 
postage, window envelopes, and business reply returns, can be 
used by researchers in higher education without seriously 
affecting the response rate . Horowitz & Sedlacek (19723 
conclude that the most efficient, least expensive method 
available for mail surveys will not s ign i f icanCi y affect 
return rates from faculty members. Thus, the literature 
tends to support many cost cutting techniques except in the 
case of follow-up. 

BesBOQ^ents/Non^resBondents 
In mail survey research, perhaps the biggest concern is 
that of response rate, Heberlein & Baumgartner (cited in 
Altschuld & Lower 1984) report that mail survey return rates 
range from an average of 46 percent from the initial mailing 
to an average of 84 percent for the fourth mailing. Yu & 
Cooper (cited in McKillip, 1984) identify the average return 
rate- for studies between 1965 and 1981 to be 47.3 percent (p. 
85) . 

Who is the respondent /non-respondent and what causes an 
individual to be one or the other? Erdos (cited in Humphries 
1983) prov ides three class i f icat i ons of respondents : 

1. Eager beavers who will answer almost anything. 
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2. People who are not interested in answering 
questionnaires, but will do so if they are 
interested in the subject, if the subject is 
important, or if there is some incentive for 
answer ing the quest ionna ire ♦ 

3. Born non-respondents who probably won't answer 
anything, (p. 14) 

Fortunately according to one study only about three percent 
fall in the third group (Robinson & Agisim, cited in 
Humphries 1983). This is fortunate because the burden of 
proof falls on the researcher to determine that there is no 
significant difference between the answers for the 
respondents and the non-respondents (Humphries, 1983). 

The question now becomes, "What factors affect t.ie 
return of a questionnaire?" Heberlein & Baumgartner, and Yu 
& Cooper (cited in McKillip, 1984) identify three factors 
that affect the return rate: "(a) Respondents' interest in 
the questionnaire topic, (b) the number of contacts with the 
respondents, and (c) the use of a small h.onetary incentive" 
(pp. 77-78) McKillip (1984) states that respondents will 
have a positive attitude toward return cf a questionnaire if 
they perceive that return will lead to positive consequences, 
and they will have a negative attitude if they perceive that 
the consequences of return are negative. Lockhart (1984) 
offers interesting fruit for thought in his six stages of 
returning behavior: 
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1. Receiving the questionnaire. The researcher must 
assure that the respondent rece i ves the 

ques 1 1 onna ire. If it is not received, it cannot 
be completed. This calls for accurate samples 
and accurate addresses • 

2 . Open Ing the ma i 1 . What en t ices the respondent 
to open the envelope? There must be some 
incentive to open the questionnaire. Some 
affiliation with a know organization would be 
helpful . 

3. Forming an overall impression. The decision 
to complete the questionnaire. Salience is 
an 1 mportant factor . 

4. Answering the questions. The questions must be 
easy to answer . 

5. Returning the questionnaire. This must be 
conven ient . 

6. Dealing with non-respondents. Reminders have 
worked effectively with the second classification 
of respondents pre v iously identified. 

Lockhart (1984) contends that some individuals will complete 
all stages within minutes, others will take longer, and some 
will never complete all the stages. Dillman (cited in Cote & 
Gr innel 1, 1984) identifies three cond i t i ons that must be met 
to maximize responses: "minimize the costs for respondents, 
maximize the rewards for doing so, and establ ish trust that 
those rewards will be delivered" (p. 11). 
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Sudman & Bradburn (1984) identify groups for which they 
think surveys are appropriate and groups for which they think 
surveys are inappropriate. They identify most special groups 
such as physicians, teachers, and accountants as highly 
cooperative in returning questionnaires. They offer three 
reasons for this cooper at ion: (a) the educat i onal level is 
usually higher and these individuals have more experience 
with forms, (b) the questionnaires usually deal with topics 
closely related to the groups' professions, and (c) a mail 
questionnaire can be more easily completed than a personal 
interview. Baumgartner & Heberlein (1984) point out that 
higher response rates can be expected from respondents who 
are in school or in the Army. On the other hand the less 
educated and the aged are populations for which surveys are 
not appropriate. These groups often find questionnaires 
difficult to read and to complete, and they are often 
suspicious of the study (Sudman & Bradburn, 1984). 

Foil, ow^ug E£2£§l£ures 

"Fol low-up contacts have been recognized for more than 
sixty years as an effective way of increasing response rates" 
(Lindsey, cited in Baumgartner & Heberlein, 1984, p. 67) 
Rigorous follow-up procedures gain stronger support in the 
literature than other techniques for improving response rates 
(Babbie, 1973; Horowitz & Sedlacek, 1974; Humphries, 1983; 
Baumgartner & Heberlein, 1984). Researchers, such as Ryan 
(1978) agree that adequate response rates are critical to the 
credibility of a study. He states that "Follow-up mailings 
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can sometimes mean the difference between having a worthwhile 
study and a worthless study- (p. 581) . Babbie ( 1973) 
suggests that a study must have at least a 50% return return, 
while others identify 80% as the minimum (Demaline & Quinn, 
1979; Hillestad, 1977; Kerlinger, 1973). 

How many f ol low-ups does 1 t take to get an adequate 
return? Futrell & Lamb (1981) find that there is no need to 
send out a follow-up at all if only one is to be mailed. The 
results of their research indicate that if the researcher is 
not willing to spend the time and money to send several 
follow-ups including additional questionnaires, they should 
not send anything beyond the initial mailing. Therefore, in 
order to gain the response rate desired in most studies, the 
researcher will need to send several follow-ups. 

Babbie (1973) states that "the longer a potential 
respondent delays replying, the less likely he is to do so at 
all" (pp. 163-164). Therefore, a well planned sequence of 
follow-ups should be implemented. 

The first fcllow-up should be oent about five days after 
the initial mailing to serve as a reminder to those who have 
not mailed the survey and as a thank you note for those who 
have (Parten, cited in Humphries, 1983). This follow-up may 
consist only of a letter or a post card. 

The second follow-up should include the questionnaire in 
addition to the reminder letter (Parten cited in Humphries, 
1S83; Rossman and Astin, 1974). Futrell & Lamb (1981) find 
that the inclusion of the questionnaire with the second and 
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third follow-up reminder letters significantly increases 
response rates over follow-up reminder letters alone* The 
second follow-up (third mailing) should be sent about two to 
three weeks after the initial mailing* 

The third fol low-up, if necessary, should be ma i led 
about three weeks after the second fullow-up (Humphries, 
1983) . Humphr i es ( 1983) further sugges ts a 1 ternat i ves to the 
third follow-up such as telephone calls, registered follow-up 
mailings, telegrams, and interviews. Comer & Kelly (cited in 
Baumgartner & Heberlein, 1984) find that telephone follow-ups 
produce a higher response rate than does mail follow-ups* 
These alternatives tend to get expensive and the researcher 
must analyse the cos t /benef i ts to using these techniques* 
But, the evidence is clear. Follow-up is the most effective 
technique to obtain the required data for a sound mail 
research study. 

£2Q£lH<?iQ2 Comments 

The popular i ty of the ques t i onna ire as a data -gather ing 
device seems firmly entrenched in the field of educational 
research (Cote & Grinnell, 1984) Even though there tends to 
be conflicting data on the response rate effectiveness for 
many techniques, it is my opinion, based on the literature 
review and personal experience, that a quality research 
instrument, a convincing cover letter, and stringent follow- 
up will result in desired return rates. 

In this paper, techniques that are unique to using mail 
survey research and many research studies that have 
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investigated and the use of these techniques have been 
reviewed. It was not the intent to discuss other research 
tasks such as analyzing the data, sampling, and research 
des igns that are also character i st ics of other types of 
research. The topics that were discussed may seem mundane 
and mechanical, but as Babbie (1973) states "The researcher 
who believes the quality of his research is solely a function 
of his analytical and theoretical abilities is sorely misled. 
A brilliant research design that is improperly executed will 
result in failure. No detail is to small or too mundane to 
be safely ignored" (p. 169). 
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DATA COLLECTION IN VOCATIONAL EDUCATION RESEARCH: 
A COMPARATIVE ANALYSIS 



THE TELEPHONE INTERVIEW AS A DATA COLLECTION TECHNIQUE 

What are the desired characteristics and outcomes sought from any data 
collection technique? Do we not seek techniques which are economical, both 
in terms of time and dollars? Do we not seek techniques which collect data 
that are both valid and reliable? Do we not seek techniques which will 
give us the most representative sample of our population by giving us the 
greatest percentage of usable responses? Do we not compare techniques in 
terms of these characteristics and outcomes? 

The purpose of this paper is to report the advantages and disadvan- 
tages we have found in the telephone interview as a data collection tech- 
nique as a result of our experiences with its use in several studies over 
the past few years. We have conducted telephone interview studies rang- 
ing from a survey of the general public of the State of Oklahoma to the 
survey of vegetable buyers in a single county. 

THE TELEPHONE INTERVIEW STUDIES AT OSU 

We first used the telephone interview to gather data for a state-wide 
study of the awareness of the general public of the components of the 
Division of Agriculture at Oklahoma State University (Cosner et.al., 1980). 
This study surveyed a sample of the entire population of the State of 
Oklahoma. Since a confidence level of .98 was chosen, this resulted in a 
sample size of 2401. After considering the size of the state, the cost of 
mailed questionnaires and the relatively low response rate expected from 
mailed questionnaires, we decided to use the telephone -interview tech- 
nique. The face-to-face interview was not considered due to the prohib- 
itively high cost of travel and the time which would be required. 



This first use of the telephone interview technique was followed by a 
study of wheat farmers' awareness of integrated pest management (IPM) in a 
four-county area of north-central Oklahoma (Finley, 1981). Again, the .98 
confidence interval was chosen, which yielded a sample size of 1556. The 
unique aspect of this study was the fact these same farmers were contacted 
twice, once to establish baseline information before the IPM educational 
program, and again at "-the- end of the program to measure impact (Key et. 
al., 1985). 



A paper presented at the American Vocational Association Convention 
for the American Vocational Education Research Association, Atlanta, 
Georgia, December 7, 1985, by James P. Key, Department of Agricultural 
Education, Oklahoma State University. 



The next three studies involved smaller areas and smaller sample sizes 
(382), since the .95 confidence level was used. The first of these invol- 
ved a follow-up study of one county out of the state-wide awareness study 
after that county had conducted an awareness program (Bergman, 1982). The 
next was a single-county study of the awareness of the 4-H Program in that 
county (Hackett, 1982). The third study was also a single-county study of 
vegetable purchase and usage patterns of the residents of a major metropol- 
itan area (Thompson, 1982). In addition, we have advised the Entomology 
Department at OSU on two other telephone interview studies involving all 
the alfalfa producers in the state. 



ADVANTAGES OF THE TELEPHONE INTERVIEW 

The primary reason for our choice of the telephone interview as a data 
gathering technique in the beginning was for the higher rates of return 
anticipated. The next consideration was lower cost, compared to the mailed 
questionnaire or face-to-face interview. Another major consideration had to 
be the time required, compared to the face-to-face interview. 

Rates of Return 

The anticipation of a higher rate of return was justified in the 
actual rates of return achieved by the different studies. The state-wide, 
general public study had usable responses from 1,662 of the 2,401 people 
contacted for a 69.22 percent response rate. The initial sampling of the 
wheat IJ>M impact study resulted in 1,194 usable responses of 1,556 contacts 
for a 76.73 percent rate. The follow-up phase of the study achieved 858 
usable responses out of contacts with 946 farmers who were still growing 
wheat, for a rate of 90.70 percent. The three studies with sample sizes of 
382 achieved return rates of 65.44 for the awareness program follow-up, 
92.00 for the 4-H awareness survey, and 74.86 for the vegetable-use survey. 



TABLE 1. 



Investigator Population Confi- Sample Respon- Non- Percent 
Size dence Size dents Respon- Res. 

level dents Rate 



C0SNER, et al. Infinite 


.98 


2,401 


1,662 


739 


69.22 


FINLEY (1981) 2,885 




(1980) 








.98 


1,556 


1,194 


362 


76.73 


KEY, et al., (1985) 




946 


858 


88 


90.70 


(Follow-up of FINLEY sampl 


e contacted, 


stil 1 growing wheat) 


BERGMAN (1982) Infinite 


.95 


382 


250 


132 


65.44 


HACKETT (1982) Infinite 


.95 


382 


351 


31 


92.00 


THOMPSON (1982) Infinite 


.95 


382 


286 


96 


74.86 
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The response rate appeared to be affected by the time called, caller's 
sex. caller's manner, respondent's relationship to the topic of the call, 
and respondent's age. Most of the calling was done between 5 pm and 10 pm. 
It was found that generally people could be contacted at home during these 
hours, with the best hours being between 7 pm and 9 pm. Female callers 
usually achieved higher response rates. However, it appeared the calling 
manner made the greatest difference, since the highest return rate (92.00) 
was achieved by a male caller. If the respondents had a relationship to 
the topic of the call, the response rate appeared to be slightly better, as 
evidenced by the higher responses from the wheat farmers to the call about 
wheat IPM. An exception to this case might be the strong return (92.00) 
from the general public to the 4-H survey. Older respondents appeared 
slightly more willing to answer the questions than the younger. Their 
calls usually took longer also as many seemed to like to chat. All of 
these surveys were designed to take from 3 to 5 minutes on the average as 
longer surveys tend to reduce response rates. 

Cost 

With a WATTS line or computerized WATTS box, the telephone interview 
is a very low-cost survey, even over a wide area. For a local area, such 
as a single county, the telephone interview costs even less. Even if 1+ 
calling is required, the telephone interview costs about the same as the 
mail questionnaire and results in a greater return rate. 

For the general public survey of the State of Oklahoma 
the actual costs were: 



ITEM COST 



1. 1,696 telephone calls from 0SU $ 552.71 

2. 705 telephone calls from the State Department $ 268.06 

3. Temporary clerical assistance/caller payroll $1,623.70 

4. Cost of questionnaire $ 120,11 



Total $2,564.58 



If 1+ dialing was used instead of the WATTS box network, the cost of 
the telephone calls would have been $1,708.44. This would have increased 
the total actual cost of the study to $3,452.25. 

It was estimated that a mailed questionnaire to the same 2401 sample 
with two follow-up mailings would have cost $3,260.59 and would have yield- 
ed a 30 to 40 percent return rate. This cost would have been quite a bit 
more than the actual telephone interview cost and slightly less than the 1+ 
dialing cost. The 30 to 40 percent response, or about half what was achie- 
ved by the telephone interview, would have doubled the cost per response. 



The actual cost of the vegetable-use survey in one county Utilizing a 
sample size of 382 (.95 confidence level) was: 



ITEM 



COST 



1. 382 telephone calls 



$123.52 



2. Temporary clerical assistance/caller payroll 



$286.00 



3. Cost of questionnaires 



$ 16.66 



$427.18 



Time 



The actual calling time from the OSU phones was logged as 5,103.4 
minutes for the 1,696 calls. This averaged approximately 3 minutes per 
call. Of course there was much other time spent preparing for those calls. 
The overall time spent preparing for and making the calls was spread over 
about two months and involved eight callers. The calling was done in the 
evenings and callers had conflicts in schedule from time to time, which 
accounted for part of the duration. It was estimated the total time would 
be about comparable for the mailed questionnaires to the 2401 sample, 
allowing for two follow-ups. It was difficult to estimate the time for 
face-to-face interviews, but would, no doubt, be quite a bit more, counting 
travel time. 



Additional Advantages 

One of the additional advantages of the telephone interview technique 
is that phone numbers are a convenient mode of Contact for respondents, 
especially where the general public is involved. There aro even computer 
programs for random selection of telephone numbers from directories. An 
additional advantage of the telephone interview over the mailed question- 
naire is the ability to clarify questions the respondent might have and to 
establish rapport with the individual. This latter advantage also allows 
the caller to establish a good image for the institution for which the 
calls are made. An additional advantage discovered with the wheat farmer 
survey was that respondents can be located through other respondents. We 
were able to locate farmers who might have moved or otherwise be diffi- 
cult to locate by asking those we were able. who lived near them. 




DISADVANTAGES OF THE TELEPHONE INTERVIEW 



One of the greatest disadvantages is the inability to sample the 
people who do not have telephones and those with unlisted numbers. Groves 
and Kahn (1976) indicated about 10 percent of the general population did 
not have telephones in 1976. The percentage has probably decreased quite a 
bit since that time. We also know from census data that those without 
phones represent the lower income population. If the characteristics of 
this group are important to a study, it will have to be recognized that 
underrepresentation may occur. Random digit sampling techniques have been 
developed to virtually eliminate the problem of unlisted numbers (Dillman, 
1978). V 

Another disadvantage is that telephone exchanges do not follow common 
geographical bounderies, such as county lines. The exception to this rule 
is state bounderies. No state lines are crossed by exchanges now. When 
trying to sample within a county, over sampling must be done to replace 
those who are selected from that exchange, but are found to live in another 
county. 

A difficulty facing any attempt to use the telephone interview is 
getting good callers. The voice quality, personality and rapport on the 
phone are most important characteristics for a caller. It was our exper- 
ience that callers with pleasing voice quality, a pleasant personality and 
the ability to establish rapport on the phone could generate a greater 
percent of responses than those with less of these qualities. 

An additional difficulty is catching people at home. As has been 
previously stated, the calling time is important. Weekday, evening-calling 
proved effective for us between the hours of 5 to 10, with the most 
effective hours between 7 to 9. We also used at least three call-backs 
before giving up on a number. After that, one of the additional numbers 
from the randomly selected oversampling pool was used to replace it, 

A disadvantage of the telephone interview is tht= limited time avail- 
able for the interview, if a good response rate is desired. We decided to 
limit our interview schedules to the amount of information which could be 
gathered in 3 to 5 minutes of straight questionning. This decision was 
based on our expectation that the response rate would be reduced substan- 
tially if a longer questionnaire were used. Compared to the face-to-face 
interview, this would limit the amount of information which could be 
gathered. 

An additional problem encountered when the general public was called 
was that there was a tendency to get more female respondents than male. No 
particular person was asked for except one over 18 years of age. Toward 
the end of the survey, the male was requested in an attempt to balance the 
male-female ratio. 
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Cautions and Precautions 



The preparation of the interview schedule is the most important phase 
of the telephone interview, as is true with any interview or questionnaire. 
However, it is our belief that with the telephone interview, it is doubly 
important. If the schedule does not have the questions worded correctly to 
gather the information needed, the calls are wasted. Also, the question- 
ing sequence must flow correctly to produce a smooth interview which will 
get maximum completed, usable interviews. The introductory remarks must be 
short, clear, yet adequately identify the caller and establish trust in 
order to obtain cooperation. 

Caller training is another important part of the telephone interview 
process. The way the caller comes across on the phone will, in most cases, 
decide whether the person is willing to answer the questions or not. We 
had callers practice using the interview schedule with- one another or with 
us, after we had a telephone company specialist hold a seminar on effective 
calling techniques. We then had them do actual calls to a pilot group 
similar to the group to be called, but who were not included in the survey. 
This thorough preparation paid off once the calling began. 

A consultant researcher made a suggestion which would help us 
establish the validity of the information we gathered on the phone. He 
suggested we run face-to-face interviews with a sample of our telephone 
survey group and compare the information gathered to establish additional 
validity for the information gathered by telephone. 



PERTINENT LITERATURE 

Three book? were found which were most informative concerning the 
telephone interview. Frey's (1983) Survey Research bv Telephone was the 
most recent located. After a broad introduction to the telephone survey, he 
described the book by chapters: "The next chapter compares the telephone 
survey with the mail and face-to-face methods on several dimensions. The 
purpose of this section is not to define which is the best method, but to 
assist the researcher in selecting the most appropriate technique given his 
or her problem, resources and timetable." Chapter 3 discusses the various 
sampling procedures that are available to those who would utilize a tele- 
phone survey. Questionnaire construction is the subject of Chapter 4. The 
problems of question wording and question order receive the most attention 
in this section. Chapter 5 outlines procedures for the administration and 
implementation of a telephone survey. This section includes a thorough 
description of Computer-Assisted Telephone Interviewing (CATI). The final 
chapter discusses ethical issues associated with telephone surveys and 
elaborates on the future prospects for telephone interviewing. 

Dillman (1978), in his book, Mail and Telephone Survey ?, did an 
excellent job of describing and contrasting mail and telephone surveys, 
while at the same time comparing them to the face-to-face interview. 
Groves and Kahn (1976) did an outstanding job of comparing the telephone 
interview with the face-to-face interview by the use of side-by-side 
surveys in the same study. Their book, Surveys by Telephone: A Nation al 
Comparison With Personal Interviews, describes in detail the advantages and 
disadvantages of both. 
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Many good journal articles were found concerning many aspects of the 
telephone interview. The more recent, plus some of the more important 
older articles are listed in the bibliography. It was found that most of 
the articles concerning the telephone interview were reported in the Public 
Opinion Quarterly. The Journal of Market ing Research was the next best 
source, with a variety of journals accounting for the remaining articles. 
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The Personal Interview as a Data Collection Technique 

By Maureen E. Kelly, Ph.D. 
University of Arizona 

Revie w of the Literature 

Like the questionnaire, the Interview can vary from a highly structured 
event to a very unstructured conversation. An example of a highly structured 
Interview 1s a telephone or person on the street Interview, where time and 
focus are of the essence. This type of Interview 1s likely to be of a 
relatively short duration and would be used when the questions are fixed, and 
one 1s seeking response from as diverse a group as possible. 

Somewhere 1n between 1s the focussed Interview, where "...the wording 
of the questions 1s not strictly specified, but the Interview 1s nevertheless 
focussed since the Information that 1s sought 1s 1n an area experienced by the 
respondent" (Adams and Schaneveldt, 1985, p. 216). This 1s the kind of 
Interview method that I have used 1n Investigating the development of 
vocational education faculty as researchers (Kelly, 1982) and one that I will 
discuss later in this paper. 

At the other end of the Interview contlnulum 1s the nondlrectlve or open 
Interview. This type of Interview, popularized by Carl Rogers, 1s used to 
provide a framework for Interaction that 1s seen as a safe setting, one 1n 
which the respondent sets the mood, tone and pace of the Interview. Such 
unstructured Interview methods are used when the subject 1s potentially quite 
wide ranging, as In Margaret Mead's work with native Samoans. 

There are a number of advantages to the Interview method. Adams and 
Schvaneveldt (1985) note that there are at least seven major advantages to the 
Interview as a research method. In addition, I have identified four 
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advantages to the specific method that I used, focussed Interviewing. 

Of the seven general advantages to interviewing as a research method, the 
first is the advantage that comes from conversing with the respondent 
personally, that of Increased cooperation. This generally results in a higher 
participation rate, since most people find it less of an effort to talk than 
to write. In addition, once the Interview has begun, rapport or trust can be 
established, which generally leads to more accurate responses. 

Second, since the interviewer has established direct contact with the 
respondent, interviewing has the advantage of allowing the researcher to 
discuss the interview and respond to any questions the respondent may have. 

Third, the quality of the data obtained using this method is likely to be 
great owing to the advantage that one gains due to the personal nature of the 
method. A skilled interviewer can "read 91 people, assess moods and probe, 
clarify, rephrase or restate questions to the respondent. Thus, Interviewers 
are free to seek information in a different way or more readily seek the 
truthfulness of the responses. 

Moreover, because this method often allows for direct observation of 
respondents, it allows for flexibility 1n the way questions are asked and for 
interviewer response to non-verbal clues, a distinct fourth advantage. 

The opportunity an interview provides to generate motivation and rapport 
1s the fifth advantage of the interview method. Since face to face 
interaction is instrumental in establishing rapport, such Interaction often 
leads to higher levels of motivation on the part of the respondent. 
Conversely, one then Introduces the potential disadvantage of needing to limit 
conversation or response! due to time or focus constraints. 

A related advantage 1s that one is engaged 1n the communication process. 
Since most people seem to enjoy talking, this becomes a real advantage of this 



method. As Adams and Schaneveldt (1985) note, essentially, the process of 
Interviewing 1s the process of communicating "...the Interview 1s very much an 
artful process, a process in which a sensitive and skilled practm oner can 
make 1t easier for respondents to use communication to forward the goals of 
scientific understanding as well as serve as a very rewarding process through 
directed conversation" (p. 215). 

Lastly, once rapport has been established, topics that are perceived as 
being too sensitive or emotional to put on paper can be talked about to a 
sensitive Interviewer. Thus, the ability to handle sensitive and enotlonal 
topics 1s the seventh advantage of the Interview method. 

Since I was Interested 1n finding out how vocational education faculty at 
research oriented universities were developed, and what enabled or Inhibited 
research productivity 1n their current position, I employed the focussed 
Interview method. Since the focussed Interview 1s designed to provide 
detailed coverage of a particular topic, four additional advantages, 
particular to this method, can be Identified. 

Familiarity with the role of the vocational educator and the "street 
knowledge" of the professional researcher turned out to be a considerable 
advantage 1n using a focussed Interview approach to Interviewing vocational 
education researchers. With such background, the Interviewer can openly adapt 
to the respondent, thereby being mo^e likely to learn about the researcher's 
real behavior, experiences, and attitudes — to become familiar with her/his 
feelings and problems. 

A second, related advantage 1s awareness of the full research situation 
and the larger communication context of the task 1t Involves. Thus, the 
focussed interview method allows looking at something 1n depth. Therefore, 
this method "..can furnish detailed narratives with fully developed readers 
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and plots thick with conflicting purposes and multiple constraints..." 
(Goldstein, 1985, p. 2), and 1s aptly suited to the experience of researchers 
In vocational education. 

Moreover, 1n my case, 1t afforded me simultaneous contact with the 
process and products of the professional who must write. By focussing on both 
the process and products of the research experience, I was able to read and 
Integrate the products of vocational education researchers with the statements 
they made about how a particular research Idea was developed. In fact, I was 
able to collect a file of models, each of which had some meaning to a 
particular researcher 1n my study and that was the result of the living 
process of research productivity. 

Lastly, because I Interviewed respondents from a multitude of settings 
and functions using the focussed Interview method, I became acquainted with a 
wide range of professional activity, thus gaining what I believe to be a 
balanced perspective on the research experience. Moreover, this particular 
Interview method is a way to keep abreast of Innovations 1n vocational 
education research. 

The Problem 

As I alluded to earlier 1n this paper, the problem that I set out to 
describe was what facilitated or Inhibited the research productivity of 
faculty members 1n vocational education. I was particularly Interested 1n 
seeing whether these factors varied by sex, vocational service area or 
academic rank. And, like many doctoral students, I had concluded from my 
courses 1n research methods that 1f you wanted to describe something, the most 
efficient and productive way of doing so was to use a questionnaire. 

However, as I was developing the questionnaire, I had the good fortune to 
have scheduled a conference with Egon Guba, the coauthor of the much quoted 
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Clark and Guba (1977) productivity studies on the education professoriate. 
Guba had come to the National Center at Ohio State to give a paper as part of 
our staff development series* 

By that time, I had developed the draft of my questionnaire and 1n the 
process of having 1t reviewed for content validity. I thought that Guba was a 
good candidate to be on that panel. To my surprise, he suggested that I should 
rethink the methodology, going Instead for a "thick" description of as diverse 
a group as I could find within the context of the research university and 
vocational education. He suggested that I attempt to Interview people to get 
a first hand account of those who were looked to as research producers - "get 
as rich and diverse a sample as you can," I remember him saying. 

Logistics 

Thus, I reached the point when I needed to rethink the methodology of the 
study. In so doing, I needed to reacqualnt myself with the advantages of this 
method and the procedures Involved 1n Interviewing. I have organized this 
section of the paper to reflect these six procedures - planning, sampling, the 
Interview guide, Interviewing, data collection, and analysis. 

1. Planning the project to weet specific objectives 

Initially, one needs to be convinced that the Interview 1s the best 
method for obtaining the desired Information 1n the study. Thus, answers to 
questions such as "What do I want to know? What will I do with the answers?" 
should clearly Indicate to the researcher and others that Interviewing Is^ the 
appropriate technique for the study. Secondly, the researcher needs to have a 
clear Idea what they will do with the Information obtained through the 
interview. 

For me, this Initial stage Involved much reading, questioning and 
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reflecting. Moreover, it brought me to the point In 1981 that I was able to 
intelligently reason and discuss the so-called quantitative / qualitative 
distinction and defend the methodology, and the purposive sampling technique I 
employed to obtain the sample. 

This purposive sampling technique was chosen to assure that a 
representative sample of highly regarded vocational education doctoral 
programs, distributed among service areas that were both sex neutral and 
traditional for their sex, were Included In the study. Thus, I chose to 
contact faculty In five vocational areas at six research oriented universities 
to participate In the study. 

Once the method has been selected, It is appropriate to survey the 
published material to see what has been done. At this stage, "the professional 
literature has a somewhat peculiar relationship to Interviewing, being both 
Irrelevant and yet absolutely essential to it - Irrelevant because the 
Information gained through the Interviewer's experience will never be found 
written up anywhere, essential because no one can interview from a naive 
position. "(Goldstein, 1980, p. 5) 

2. Selecting and contacting organizations or respondents 

In any study, selection of respondents 1r usually a compromise between 
accuracy and cost. This 1s particularly true with interviews, as time, 
travel, and transcription costs can quickly mount up. In my case, 101 
potential participants were contacted by mail to determine their willingness 
to participate 1n the study. Of the 101 faculty members contacted for 
participation 1n the study, 86 agreed to be Interviewed, for a response rate 
of approximately 85%. 

I believe that I obtained such a high response rate for two reasons. 
First, my initial contact letter was one that gave a full statement of the 
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project's purpose and format, the amount of time allotted to the Interview, 
what, 1f anything, you are requested from them and the pledge of 
confidentiality 1n the reporting of the Interview data. 

Secondly, once I arrived at the particular university and began 
Interviewing, I made a concerted effort to follow up on those who had not 
replied to my Initial letter, and 1n some cases, to those who had refused to 
be Interviewed. I found that once I became known around the. department, 1t 
was relatively easy to use successful Interview respondents to Introduce and 
promote participation 1n the study to others. I suspect such tenacity 1n an 
Interviewer enhances the participation rate. 

On the matter of scheduling Interviews, I have two tips: 

1) 1f you ask for 45 minutes, you are likely to get an hour, due to the 
Interest of respondents once engaged 1n conversation and the way time 1s 
usually scheduled. 

2) Try to use conventional organizational techniques such as sex or status to 
schedule Hke respondents 1n similar slots as you go from place to place. 
Upon reflection, I wish that I had systematically slotted Interviews with 
department chairs at the beginning or end of the week, or those from Hke 
service areas 1n close proximity to one another. 

3.Construct1ng the Interview guide 

Although no Interview will proceed exactly as planned, detailed planning 
will help the interviewer get the Information with a minimum of error. The 
format of the Interview should be such that the majority of the questions are 
open-ended, so as to get a free response. Moreover, they must be organized 
Into a conversational structure to allow the interviewee an opportunity for 
full, accurate answers. Hopefully, a well organized Interview will be 
stimulating and worthwhile for the respondent, without provoking anger or 
hostility. Any sensitive or potentially threatening questions should be 
organized in such a way as to be posed after a Hne of questioning and its 
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purpose has been established* By waiting until the respondents Involvement 
has been encouraged, the Interview has reached that one 1s likely to tell the 
truth even 1f 1t 1s damaging to oneself* 

During the Interviews that I conducted, an experience checklist helped 
develop a frame of reference for the Interviewee at the beginning of each 
Interview session and gave me an Idea as to where I might start our 
conversation. In addition, I found that simple warmup questions such as where 
the respondent was from and how their vocational aspirations were developed 
seemed to put the respondent at ease and provided an personal dimension to the 
interview that was helpful. 

4. Conducting the Interviews 

A prerequisite to establishing rapport 1s a suitable meeting place with 
sufficient privacy to get detailed, honest answers. In most cases, my 
interviews were held on campus 1n the participant's office. Occasionally, 
scheduling Interviews at professional meetings was required, and 1n these 
cases, this meant that I had to meet with people 1n restaurants, lobbies, and 
hotel rooms where (I suspect) people felt less comfortable than they might 
have felt 1n their own offices. However, with some sensitivity to possible 
Intrusions or distractions, even a public place can br suitable. In my case, 
it served as another way of Increasing the participation rate. 

The opening of the discussion should be one that Inspires confidence and 
Interest. It 1s appropriate to briefly Introduce oneself, and follow this by 
forecasting the purpose, method, and topic of the interview. 

Motivating the respondent to become Involved 1s the next step, for 1f 
stimulated by the discussion, the Interviewee tends to overcome their initial 
reserve or anxiety and begin to voluntarily offer comments and suggestions. 
My topic seemed to have been a natural to motivate discussion, for every 
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faculty member Involved 1n research seems to have some opinion or experience 
to share. Many people told me during the Interview that they hadn't planned 
on telling me what they really thought Inhibited research productivity, but 
once they got talking, 1t seemed to roll right out. 

No one Interview method will work with everyone, so an interviewer needs 
to be sensitive to each personality, and adapt his/her style to suit the 
Individual. I have to admit that this 1s where I believe that I had the 
greatest problem 1n my study, for I found that Interviewing such well known 
vocational educators sometimes left me at a loss for words, to say nothing of 
Interview technique. 

However, where there was good rapport, 1t allowed me to probe for answers 
and seek corroborating evidence on complex Issues, either between or among 
respondents 1n the same field or within the same department. 

One method that helped me achieve this corroboration was the method of 
data trlangulatlon used in the study. In my study, both quantitative (such as 
counts of publications reported on vltae) and qualitative (personal Interview 
and experience checklist) methods were used. Guba (1981) states that the use 
of multiple methods and trlangulatlon of observations can add to the rigor of 
a study. ' While the context 1n which Guba refers to data trlangulatlon 1s 
primarily qualitative, I believe that this added tremendously to my study, one 
that was completed at a time when the Interview as "a way of knowing" seemed 
to be under close scrutiny. 

5. Recording and analyzing the responses and observations. 

The method I used to record my data was to audio tape each interview and 
take copious notes during the session. Of course, one needs to obtain 
permission to tape record prior fco the Interview. According to Freeman (1980) 



ERIC 



9 

43 



sometimes reaction of subjects to tape recorder Is a problem. However, being 
somewhat of an audiophlle, I had used this study as an excuse to Invest In a 
Sony Walkman style of tape recorder when they were just starting to appear on 
the market. This turned out to be quite an advantage In obtaining permission 
to record, since It was an Interesting novelty to most respondents and was 
quite unob :rus1ve. 

Here are some other tips on recording: 

1) Quality 90 minute tapes are worth the Investment for two reasons: they 
come In hard plastic protective cases and hold up much better In the long run 
and there Is distinct psychological value In not having to stop to turn the 
tape over during the Interview. In my case, when the tape ejected after 45 
minutes, I was usually at a point In the Interview where I could wrap It up or 
ask for more time. In addition, any space you might have on the second side 
can be used for another Interview or (where there Is minimum space}, you can 
record your own comments after the Interview or during analysis. 

2) Always carry extra batteries and tapes fn case either falls and protect 
your equipment from temperature excesses. I had one Interview that could not 
be recorded due to simultaneous tape and battery failure and two others that 
were delayed somewhat due the excessively cold temperatures that morning. 

Thleman (1980) encapsulates the challenge of recording and guiding the 
Interview in the following manner: u \h2 relationship between the Interviewer 
and the subject Is at times very Intensive. During these moments the 
Interviewer has to face the difficult demands of being able to concentrate 
simultaneously on having an accurate overlapping control of the situation and 
retaining a deep Involvement In the ongoing process of mutual understanding" 
(p. 16). Thleman goes on to suggest that during the Interview, one should 
give full concentration to the dialectical process. During the analysis, the 
researcher's responsibility Is to give a proper analysis of the text given. 
Thus, the Interviewer would have to analyze herself/himself as one of the 
parts in the written dialogue. 

One way to Incorporate the dialectical process into the Interview is to 
use what Patton (1980) terms a successive focusing method. In the case of my 
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study, this means that the Interviews were conducted 1n clusters of two 
Institutions at a time (33% of the sample), with analysis of Interview 
transcripts and categorization of the data Into theme areas completed at three 
separate times. As subsequent Interviews were conducted, Interview questions 
were restated or additional questions were added to the Interview* As 
categories developed, they were verified by a second person off-site and 
presented to subsequent respondents for confirmation checks. 

So far as the analysis procedure 1s concerned, this is where the major 
difference between qualitative and quantitative procedures seems to end. 
While those who have quantitative data to code and process using the computer, 
Interview data 1s "... an unwelldy mass of bits and pieces — overlapping, 
Incomplete and often unclasslflable "(Goldstein, 1980, p. 9). Therefore, 1t 
makes good sense to acknowledge that the Interviewer 1s part of the 
Instrumentation and must be constantly M f1ne tuned." Thleman emphasizes that 
M ... we must m*ke use of our capability as human beings to respond to and 
understand other human beings i.e. the respondent 1n our Interviews. In our 
talks, the situation, with Its expectations and fulfillments, constantly 
alters and this must be taken Into the dialogue * (Thleman, 1980, p. 6). 

In my case, the Interview transcripts of the resulting group of high 
and low producers were subjected to content analysis. Following the method 
outlined by Berelson (1954), two major units of analysis were used: words and 
themes. Therefore, when specific words were used repeatedly 1n Interviews, 
such as Inhibitor or enabler to research, they were recorded and categorized 
accordingly. Similarly, when like propositions or themes emerged — I.e., 
"research ambience" or "research ethos," they were grouped together. 

The results were reported using the inductive analysis procedure describe 
by Patton (1981) as logical analysis, this means that the Investigator 1s 
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looking for emergent patterns In the data. As Cuba (1981) views It, there 
should be Internal homogeneity and external heterogeneity. Internal 
homogeneity refers to the extent that the data hang together or dovetail In a 
meaningful way. External heterogeneity refers to the degree to which 
differences among categories are bold and clear. 

Problems and Successes 

Holcolm talks about the pitfalls of the Interview as follows: "The 
moment you turn off your tape recorder, say goodbye and leave the interview, 
It will become Immediately clear to you what perfect question you should have 
asked to tie the whole thing together ... but d1dn c t M (In Patton, 1980, 
p. 295). Therefore, I feel that I should discuss the pitfalls of Interviewing 
prior to concluding. 

As I see It, there are three major pitfalls to Interviewing - Interviewer 
anticipation, Inexperience, and probing too far. 

The first, Interviewer anticipation, Is Implied by Holcolm In the above 
quote. A major problem In Interviewing Is that the Interview situation Is 
much like any other type of communication - It encompasses a good deal more 
than what can be transcribed from the tape recorder. By that I mean that 
Interviewers continually remark that what they knew had been covered during a 
talk did not appear In the transcription. Thus, Interviewers conclude that It 
was an assumption they made by the sequencing of comments or the shared 
meaning, expressed verbally or nonverbally, between respondent and 
Interviewer. 

Interviewer Inexperience Is a relatec 4 problem. Often, when one reviews 
transcripts, one notes that the Inexperienced interviewer wanted a particular 
answer so much that s/he took the lead when the trigger word or phrase was 
uttered. Unless the Interviewer realizes this and Immediately feeds this 
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occurrence back to the respondent for discussion, one 1s left with a shakey 
foundation upon which to make a research conclusion. 

In wanting the Individual to express as much as he/she knows about the 
topic, we occasionally go beyond the point where the subject knows any more. 
Thus, the last pitfall of Interviewing, probing to far, 1s seen. In 
Interviewing, the Ideal model 1s get the respondent "... to express as much 
as possible, to add some new facts to the body of knowledge previously known, 
reflect on the total description, and finally to comment on the whole 
process" Thleman 1980, p. 10). When the Interviewer goes beyond the point 
where the respondent 1s ab\e to contribute, the Interview momentum tends to 
stop, making 1t difficult for a satisfactory conclusion to be made to the 
process. 

Conclusion 

It should be evident that the interview 1s a useful and workable research 
method. Moreover, I hope that I have provided the reader with some workable 
tools in thinking through the interview setting, skill and training of the 
interviewer, openness and frame of the respondent, the subject under study, 
and a host of other situational factors that enter into the process of 
obtaining data via the interview. 
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Introduction 



Quality research findings of any particular study largely depend on 
the data collection procedure. This is especially the case with surveys 
and other non-experimentaJ research designs. Specifically, the technique 
employed to generate research data is of prime significance to the quality 
index of a given study. 

In vocational education research, which has been dominated by survey 
designs, (Mannebach, 1980; Kahler, 1973; McCracken, 1970), researchers use 
a variety of data collection techniques. The three principal techniques in 
common use have been the mailed questionnaire, personal interview and 
telephone interview. In agricultural education, for example, the mailed 
questionnaire as a data collection medium is by far the most dominant 
technique (Mannebach, Mckeena & Pfau, 1984). 

There are merits in the use of each of the data collection techniques 
as researchers would submit. For example, Orlich, et al. (1975), Som 
(1973), and Raj (1972) outline the advantages of the mailed questionnaire 
as a technique. The strengths associated with the use of personal 
interviews have been extensively studied by Cannell, Oksenberg, and 
Converse (1979) and Weissman (1981). Also researchers have employed the 
telephone as a primary medium to collect data. Its use has been advocated 
over others, particularly personal interviews, by Finley and Key (1983). 
Miller (1983), Dillon (1979), and Groves and Kahn (1979) also recommend 
its use. They argue that it helps reduce error variance, which could 
result from respondents' evasiveness of highly sensitive issues. 

However, each of these three commonly used techniques, like all other 
research procedures, has attendant weaknesses or drawbacks. Frankel and 



Dutka (1983), MacLean and Genn (1979) and Meyer (1971), to mention only a 
few, point to the serious disadvantages in using the mailed questionnaire. 
Obvious concern for data reliability, particularly in very sensitive 
issues, resulting from the use personal interviews has been expressed by 
Cannell, Oksenberg and Converse (1979) and Weissman (1981). As to the 
problems with the use of the telephone to collect research data, Sudman 
(1981, p. 1) writes "There may be slightly higher refusal or don't know 
responses ad shorter answers Because respondents 1 suspicions may be 
higher while motivations to talk are not as great." 

At this point, the question that could be asked is: Is there yet 
another alternate data collection medium that could be useful to vocational 
education researchers to accommodate certain specific circumstances? 
Obviously, the answer is in the affirmative. The audio cassette tape has 
been employed as a research device to collect data. Needless to say it has 
been used with great success in other educational activities. 

A number of studies on the impact of different methods of feedback to 
student performance suggest that the audio cassette tape appears to be a 
very useful tool (Hurst, 1975; Kahrs, 1974; Miller, 1973; Moore, 1977). As 
Moore (l l >77, p. 9) concluded "The greatest advantage of using tape recorded 
feedback is in the saving of faculty time." 

In a more recent study (bin Yahya, 1985), the audio cassette tape was 
proven to be the most appropriate technique in data collection under the 
given circumstances. Dubridge (1981) also pointed out three merits of this 
medium as (1) control or directness, (2) informality, and (3) flexibility 
of communication. 



Empirical research on the efficiency of this medium relative to the 
other three commonly used techniques is yet to be undertaken. This forum 
is a humble beginning for the much needed dialogue in this regard. 
Additionally, it could bring about efforts directed at such comparative 
studies. 

The Problem 

For the purpose of the symposium, the principal problem is an 
investigation into the effectiveness of using the audio cassette tape to 
collect research data. It should be pointed out that this was only a side 
study, part of a completed doctoral research by this author. The primary 
purpose of that study was to identify the role of the United States 
agricultural teacher education profession in international agricultural and 
rural development. 

The doctoral study in question was designed as a complete census 
survey. It was to accumulate information pertaining to over 30 years of 
involvement of 117 agricultural teacher educators in international 
development activities. A principal goal of the study was to prepare a 
complete and comprehensive description of this involvement. Thus, this 
researcher was expecting very large volumes of information. 

Consequently, the researcher needed a data collection technique and 
medium that would facilitate the gathering of the large amounts of 
data. Subsequently, and with the primary goal of the study in focus, the 
audio cassette tape with a mailed questionnaire accompaniment was selected 
as an appropriate technique. 



ERIC 58 



4 



The Technique 

Data collection was accomplished by using two media. The primary 
medium was audio cassette tape which was mailed to each of the 117 
identified teacher educators along with the research instrument and a cover 
page with directions and examples for responding to the 19 questions. The 
educators were asked to respond to questions by verbally recording their 
responses on the tape. 

Rationale for Selecting the Technique 
Nine factors dictated the need to use the audio cassette tape 
for this study over any of the three other techniques. 

1. Qualitative Data Bases ; Items on the research instrument based on 
the objectives of the study principally addressed qualitative 
variables. Therefore, it was inappropriate to resort to 
forced-choiced questionnaires or check lists alone. It required 
soliciting qualitative information. 

2. Diversity of Experiences ; Previous studies (Meaders, 1983; 
Thuemmel McCreight and Welton, 1983) indicated that some of the 
members of the population have been involved in international 
development activities for more than a decade. Many of them had 
very wide range of experiences by type of assignment, activity, 
and geo-political location. A data collection medium was needed 
to tap both the richness and diversity of those experiences. 

3. Completeness and Comprehensiveness of Information Base ; It was 
the purpose of this study to generate as completely and 
comprehensively as possible the past experiences of the members of 
the U.S. agricultural teacher education profession in 
international development. In addition, there was the need to 
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identify the future role of the profession in the same regard. 
Definitely, a forced-choice or even open-ended questionnaires 
requiring written responses would have failed to tap vast amounts 
of the information. Telephone or personal contacts would have been 
possible but cumbersome (and costly. . .referred to below). 

4. Full and Free Disclosure : A medium or technique was needed to 
allow and ensure free, uninhibited yet full disclosure of all 
experiences arising from involvement in foreign assignments. 
Respondents were not going to be placed on the spot or in an 
awkward situation to respond to items. Likewise, they were not to 
be coerced to respond to, for example, a forced-choice 
questionnaire. The free-disclosure principle according to 
Bijksti* and Van der Zouwen (1982), Groves and Kahn (1979), and 
Dillman (1978) permits respondents to provide complete and 
detailed information. 

5. Directness and Control of Communication : The selected data 
collection technique was to allow respondents to be able to adjust 
or "manipulate" the "response environment" to suit their own 
schedules and circumstances. Durbridge (1981) pointed out that 
one of the merits of the audio cassette tape in information 
gathering or dissemination was the ability to direct and control 
the communication process. 

6. Informality of Communication : Respondents were to be offered an 
opportunity not only to direct and control the "response 
environment" but also to feel an open, informal atmosphere as 
well. Again Dubridge (1981) identified informality of the 
communication process as an advantage of this technique. Feeling 
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more relaxed while responding to a question or addressing issues 
would very likely encourage full and free disclosure of 
experiences. 

7. Size of Population : Except for the use of a short-form check list 
research instrument, use of any of the other techniques to 
generate the kind and amount of information that was expected from 
117 respondents would have been very cumbersome. It is very 
likely z\\r. uhe response rate, and completeness of and detailed 
responses would have been compromised if any of the three other 
techniques were employed. 

8. Relative Cost : With a meager budget of less that $2000.00 and a 
population of 117 members, it was economically prudent to select a 
technique that would be inexpensive yet help achieve the primary 
goal of the study. The number of telephone calls to one 
respondent in order to make one "successful contact" could average 
more than one. Also, the amount of time to be spent with each 
respondent could be an average of 15 minutes barring poor 
connections and interruptions. Personal interviews would have 
required, in addition to "interview time", travel and other basic 
expenses to 47 of the 50 states. 

9. Personal Touch Factor : This researcher recognized that so much 
was demanded of the 117 teacher educators. As a result, he needed 
a data gathering technique through which the educators could 
express their true inner feelings such as being concerned, 
cheerful, friendly, the supportive type, or the less concerned, 
distant, or almost cold type. Listening to the individual voices 
was by itself a special type of experience the researcher 
anticipated. 
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VALIDITY AND RELIABILITY ISSUES 

Research on the methodological issues on use of the audio cassette 
tape is virtually nonexistent* Literature on the use of the medium in 
education particularly as an instructional material (Hurst, 1975: Miller, 
1973) and in professional preparation development (Kahrs, 1971) is not 
abundant* 

A review of available literature reveals that no study has addressed 
the issues of reliability or validity of data collected with the cassette 
tape. This researcher did not compute the reliability coefficient for the 
data gathered in this study. The decision not to determine the coefficient 
was for two reasons. First, the bulk of the information generated was 
highly qualitative consequently, it was difficult to subject the data to 
such statistical manipulations. Second, there is yet to be a formula for 
statistically computing reliability coefficient for qualitative data. In 
short, there is no such theoretical or computational base for the 
practitioner, for according to Miles and Huberman (1984, p. 20), "There are 
few agreed on canons for analysis of qualitative data." 

Another reiteration at this point is in order: Information from the 
study, in question was basically qualitative in nature. Responses were 
provided verbally in 77 of the 96 usable cases. 

From a practical standpoint and within the context of this study, a 
central issue on reliability, defined conceptually in terms of consistency 
or repeatability of results (Zeller and Carmines, 1978), needs to be 
considered. The question is: Would the teacher educators provide the exact 
responses time and again if the same research instrument was used? It is 
doubtful that their semantics would remain the same. But the basic 
information of interest to the study would probably be provided. 
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Though it is highly likely that the educators* responses would be 
consistent over time, research methodologists tend to be critical of the 
credibility of such qualitative data. Not surprising therefore, questions 
have been raised about the "natural validity" (Kennedy, 1984, p. 367) of 
such data. Her key concerns centered on four issues: (1) obstrusiveness of 
the inquiry itself, (2) reliance on verbal testimony of respondents, (3) 
the inherent ambiguity of the language of inquiry and responses, and (4) 
"much of the testimony gathered is essentially hearsay evidence," The last 
concern does not seem to apply to this particular study. Thus, 
she concluded: "the quality (my underline) of the investigator's data 
depends on the quality of the participant's testimony; testimony that is 
shaped not only by their concern for social desirability, but also by 
things such as their insightf ulness, their articulateness, and their 
openness" (p. 367), 

From the foregoing paragraphs, it appears the problems of data 
validity and reliability need not necessarily be directly correlated with 
the use of the audio cassette tape as a data collection technique. 
Nonetheless, it would be appropriate to suggest that research 
methodologists, particularly those with great interest in qualitative data, 
conceptualize and formulate models for data quality analysis. A need 
exists to arrive at what Miles and Huberman (1984, p. 20) referred to as 
"agreed-on canons for analysis of qualitative data." 

Data Manager ;nt 

Since the study was a complete census and the data basically and 
largely qualitative, it was inappropriate to use inferential statistical 
procedures for analysis. Consequently, only frequency counts, a 
descriptive statistic, was employed. 
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Responses were provided on two principal media: the audio cassette 
tape (n«=77) and sheets of paper (n c 19). This paper focuses on the handling 
of data from the first of the two media. 

First, each tape was played back by the researcher. The information 
was transcribed as completely as possible one question after another. 
Since the credibility of qualitative data is usually in suspect, Adams 
(1981) suggestion that researchers focus on only the critical Issues 
arising was heeded by this researcher. Secondly, and consequently, data 
for each question were then analyzed to identify commonalities, eight of 
which represented the specific objectives of the study. Finally, following 
Burstein's (197i>) idea that data aggregation is a viable option for 
analyzing massive databases, the analyzed data were aggregated beyond the 
individual level. 



Respondents were offered an opportunity to react to this data 
collection technique. The final item on the questionnaire requested free 
contribution to the debate in the use of the cassette tape in this study. 
Responses to that item are presented in Table 2 (negative and cautionary 
comments) and Table 4 (positive remarks). The question was: 



It is obvious that there are both actual and potential problems 
associated with use of audio cassette tape in any type of educational 
endeavor. Miller (1973) identified two specific drawbacks when he used the 
audio cassette tape as an instructional Tool. These y he noted, were 
monotony of listening and problem of rapid random access to specific 
responses. 



PRELIMINARY RESULTS 



"Would you have preferred providing this information 
using some other method? Please be specific." 



Drawbacks 
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Hurst (1975) cautioned users of the medium on three key issues. The 
first relates to the listening time when the tape is played back* Second, 
there is the fact that someone (in this case a potential respondent) may not have 
access to a recorder to provide responses. Finally, he noted the 
difficulty with shortage of magnetic material such as the tape for any long 
periods of time. 

The specific experiences of both this researcher and the respondents 
regarding drawbacks to the technique are presented in Table 1 and Table 2. 
Table 1 is a summary of experiences and Table 2 represents testimonies of 
respondents. 
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Table 1 

Actual and Potential Drawbacks in Using the Audio Cassette Tape as 
Data Collection Medium* 



The Researcher 



Respondents 



Actual 



1. 



2. 



3. 
5. 



Time consuming transcription 
phase; an average of 50 
minutes for each of 77 tapes 
Difficulty with data analysis j 
time to analyze and identify 
commonalities was long 
Difficulty with deta evaluation 
since data bases were massive 
Difficulty with data summary 
Untenable assumptions: 

a) that the cassette tapes 
were usable (n«2) 

b) that each potential respondent 
had access to recording equipment 
(n«2) 



1 . Time-consuming 
recording process 
(n-6) 

2. Difficulty in re- 
viewing responses 
(n«l) 



Potential 
1. 



Difficulty in interpretation and 
use of data (n*5) 

a) Poor transcription could compromise 
any attempt to directly quote 
respondents 

b) problem with possible ambiguity 
of the language of responses 



1. Inability to 
express self 
clearly and easely 
could lead to 
(n«3) 

a) waste of tape 

b) limited useful 
inf onration 



Note: The number in parentheses indicates the frequency of reactions 
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Table 2 

Reactions of Respondents to the Use of Cassette Tapes as the Data 
Collection Medium in this Study: Negative and Cautionary Comments 



ERIC 



1. "Easier using a questionnaire with a checklist." 

2. "Would have preferred writing." 

3. "On this paper, saved time" 

4. "The only problem is one cannot easily review the responses that 
one has made." 

5. "I prefer the method I chose. "(He responded on the questionnaire). 

6. "However, there will be probably some concerns if some quotes are 
taken out of context then used in a thesis. Normally, I would 
prefer putting down in writing anything that might be quoted in 
exactitude. If quotes are misused it could cause later problems." 

7. "One thing I might be doing is just rambling on and not giving you 
the information that you need." 

8. "You as the researcher is going to have a problem sorting out 
what* 8 relevant and what isn't from the conversation." 

9. "I think this might provide some difficulty for you in 
determining what you want to write out of this." 

10. "T need to organize and write my responses. The tape recorder is 
time-consuming for me to use. I'd rather have my responses in 
writing so that I can see what I'm really presenting to you and 
not use a lot more words that what I'd really use. Although using 
a tape recorder is a good idea ard is one way of hearing the 
persons voice as they relay their ideas and experiences to you, 
one of the major problems of using a tspe recorder for a study of 
this type is that you have to interpret what the person filling 
out the research information form is really trying to say." 

11. "Yes. Your tape was bad for some reason. I lost patience in 
working with it and gave up. My time is too valuable to be wasted 
on poor tapes. My written responses were done in 15 minutes. 
Sorry about your desire to have this on tape. Sometimes our best 
plans don't work out." 

12. "I'm not sure. . A little frustrating to go through the 

countries one after another." 

13. "The questionnaire developed to solicit the information on this 
project is very much open-ended and seems to be designed to hunt 
for general interesting information rather than specific questions 
other than the generalized ones of this particular study." 

Table 2 continues 
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Table 2 continued 



14. "It required that I get a tape recorder to work to make sure that 
it was operating before I completed this questionnaire. If I were 
to answer the questions by handwriting or questionnaire, I would 
have probably gotten this back quicker to you." 

15. "I would have preferred to have typed this rather than to have 
used the cassette/ 1 

16. "I guess I'm a traditionalist. A pencil-and-paper questionnaire 
would have resulted in quicker response* at least for me." 

17. "Sorry, but I was simply unable to use the cassette as suggested. 
I hope this written format is acceptable. 11 

18. "I think it would be very difficult to summarize the information." 

19. "Yes. I feel that I would have been much more effective if I 
would have been able to express my thoughts on paper as opposed to 
tape. Doing this was very time-consuming," 

20. "Yes. I guess I would have preferred check-off kind of response 
since the responses that I made would not have been what you b«d 
in mind." 

21. "I think that a regular forma': would have been just as good in 
this case." 

22. M As for providing the information, I might have been able to get 
the information to you quicker, had I not had to arrange for a 

tape recorder and take the time to record it. . (Now, that may 

simply have been my problem rather than the research method)." 

23. "However, I do not envy you in your evaluations." 

24. "Preference option taken. Sorry." (Note : He responded on paper r) 

25. "I believe it would be a gocd way to respond but perhaps more 
difficult to analyze." 

26. "Yes. My tape was jammed in such a manner that I have not been 
able to play any of it." 

27. "This method is fine." (Note: He wrote his responses.) 

28. " • I feel that probably using a tape could be effective in some 

other kinds of study. But I felt that this kind of questionnaire 
I could have been more effective if I had been able to express it 
in writing." 
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Merits 

Previous studies indicate the audio cassette tape has some merits. 
Kahrs (1971), for example, believes the medium facilitates comprehensive 
and informative evaluation. Hence, it is an effective feedback technique. 
He further points out that "The cassette tape not only allows personal, 
live interaction but it encourages one to evaluate in its total perspective 
the assignment undertaken" (p. 160). 

According to Miller (1971) the tape technique has four major 
advantages as an instructional material. Three of them directly relate to 
this study. They are the relatively low cost, compactness and convenience, 
and relatively rapid rate of speaking. 

Hurst (1975) notes three merits of the use of this medium. The first 
of these is the explicative or the ability to explain in detail. Next, it 
is a time-saving device for the user . Finally, there is the ease factor 
for verbalizing responses. 

Table 3 is a summary of the specific merits related to this researcher 
and the testimonies of the respondents. The number in parentheses 
indicates the number of testimonies. Listed in Table 4 are testimonies of 
some- respondents regarding the merits of the technique. 
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Table 3 

Actual and Potential Merits in Using the Audio Cassette Tape as Data 
Collection Medium 



Researcher 



Respondents 



Actual 



1« Accumulation of wealth of information 
spanning over 30 years of multitudes 
of activities by 77 teacher educators 
in over 60 different countries 

2. Efficiency: compactness and convenience 
of the (120-minute) tape; compare it 
to the use of other medium for the 
same kind and amount of data 

3. Cost effectiveness: collecting data from 
the 77 educators via any of the other 
three techniques would have been very 
expensive 

4. Personal, live interaction is facil- 
itated 



1 . Time-saving : Some 
respondents (n*8) 
felt the technique 
was a time-saver 

2 . Free-disclosure 
facilitated: a num- 
ber of respondents 
(n«3) said the 
technique allowed 
them to freely dis- 
close information 
in detail 

3 . Thought-provoking : 
in the opinion of 
one respondent 
(n«l) the method 
permitted me to 
think much deeply 
before responding 

4. Novelty: The tech- 
nique was perceived 
by some respondents 
(n«10) as a unique 
approach 

5. Informality of re- 
sponse environment 
(n-4) 



Potential 



2. 



Reduction in response sets 
typical of respondents 
confronted with critical, 
sensitive issues 
No problem with response 
variance in this particular 
study since experiences 
varied greatly. 



1« Control or direct- 
ness: "manipula t e" 
the entire infor- 
mation-giving 
process 

2. Flexibility of the 
communication pro- 
cess on part of 
each respondent 



Note: The number in parentheses indicates the frequency of reactions 
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Table A 



Reactions of Respondents to the Use of Cassette Tapes as the Data 
Collection Medium in this Study Positive Comments 



1. "I like it. It'a rewarding; permitted deeper thinking 11 

2. "Much quicker" 

3* "Most pleasant way to respond to any inquiry" 

4. "My time was saved" 

5. "It's alright. I would probably have written more than I said" 

6. "So far as I'm concerned, this is a very good way to get a lot of 
information" 

7. "I wouldn't have taken the tine to have written all this in long hand" 

8. "I think thia worked Okay" 

9 "This method of information referral is certainly satisfactory 
with me" 

10. "If I had a whole lot of to say* this might have been the easier 
method" 

11. "I have no problem with the tape recorder* It's a unique idea. 
I'm happy to do it." 

12. "I must admit thia ia a very novel but time-saving way of providing 
this information" 

13. "This is alright" 

14. "I think thia method is quite alright" 

15. "This method appears to be a useful nne for providing for these types 
of studies. I have no problems with utilizing the tape in giving you 
my response." 

16. "It haa loosened me up to say certain things I probably wouldn't have 
said if I had to write them down." 

17. "No problem for m*.- . You probably have got a lot than you would 

have if it had been written. I like it." 

18. "This method of collecting data was quite acceptable." 

19. "No. I feel comfortable in using the tape. • I think this is a 

novel idea to use the tape and make a recording rather than write 
it down. It definitely did save me time." 

20. "This suits me juat fine." 

21. "No. I think you had a good idea in using a tape recorder for your 
responses." 

22. "No. Thia was quite quick." 

23. "Method, O.K." 

24. "Heck! This was a real treat. I enjoyed responding over the tape 
recorder." 



Table 4 continues 
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Table 4 continued 



25. "This method is fine once I took the time to sit down and put it on 
tape." 

26. "The tape situation is O.K. I have no problems with it." 

27. "Probably no. This way was O.K." 

28. "No. This method of providing information was quite adequate." 

29. "I f m enthused about this method. I hope that my tape is 
understandable.— — . But I think it is a fine medium. I've 
enjoyed responding by tape." 

30. "The format of the tape is fine. I'm sure that you can get a lot more 
information as long as you can transcribe it and understand what I am 
saying." 

31. "I do appreciate this method of data collection. And because of that 
I do want to cooperate fully." 

32. "Okay. (However, it provided opportunity to ramble.)" 

33. "O.K." 

34. "I think in terms of my own case, this is a good medium." 

35. "I think this is a good approach. It is unique. I have used it once 
in working with a blind doctoral student. I gave him all of his 
questions on one tape. He answered them on another. It worked out to 
his satisfaction and ours too." 

36. "it was a different approach. It was a unique approach." 

37. "This is an innovative way to provide the information. I expect to 
become more comfortable with it over time." 

38. "My response is generally positive. I sometimes get tired of drafting 
memos and writing letters. This is a long questionnaire. This method 
of responding was probably more suitable than another writing 
exercise. I appreciate the cassette tape and the chance to respond 

ip. this way." 

39. "I think this is a very efficient way to get information. If a person 
writes out his/her answers, they might be a little more condensed. 

But then again, they are less likely to do it." 

40. "I find this method very unique. I have used something similar to 

this in trying to get a handle on what constitutes good teaching.- . 

This reaffirms my idea on that. This is an effective way of 
communicating. Research results can be compiled from tapes." 

41. "This is a unique idea in using the tape. I congratulate you on it." 
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TENTATIVE CONCLUSIONS AND SUGGESTIONS 



Conclusions 



2. 



1. 



2. 



Researchers dealing with highly qualitative data would find the use of 
the audio cassette tape as a data collection medium quite useful 
especially if they expect to tap vast amounts of inf oraation. 
Successful use of the technique appears to depend largely on adequate, 
detailed planning in advance on how to conduct the data transcription 
and analysis phases. 



The technique would yield satisfactory results with one major 
precondition: if it is determined that members of the population have 
the ability to verbally express themselves eloquently. Under such a 
circumstance the researcher should consider use of the method. 
Other researchers employ the technique where appropriate to determine 
whether the above preliminary merits and drawbacks as experienced by 
both this researcher and his responding population could be supported 
or shown to be a series of isolated experiences. 

Comparative analytic studies be conducted on the relative advantages 
and problems associated with all four techniques under discussion here 
audio cassette tape, personal interview, telephone interview, and 
mailed questionnaire. 
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Research need to be undertaken to address use of the audio cassette 
tape technique in relation to such specific issues as: 

a) non-response rate: the average lower limit of which Lindstrom 
(1983) believes is between 5-10 percent; th<; question of 
particular importance would be: Is there a consistent deviation 
(above or below) from this limit? If so what factors are 
responsible for the changes? 

b) non-response type research questions to be considered would 
include: What is the minimum proportion of the item non-response 
type? refusers? non-accessible persons? 

c) characteristics of non-respondents: efforts should be directed 
at identifying those characteristics that distinguish 
non-respondents from respondents; also to delineate the 
differences among the various types of non-respondents. 
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